Digital signal encoding and decoding device and method

ABSTRACT

An encoding device includes: a difference generation unit for obtaining a first pixel difference value as a difference value between a first pixel value and a pixel value of a pixel having the same color as the first pixel positioned in the vicinity of the first pixel; a quantization width decision unit for deciding a quantization width in data generation by quantizing the first and the second pixel value according to the number of digits of an unsigned integer binary value of the first pixel difference value and the number of digits of an unsigned integer binary value of the second pixel difference value generated in the difference generation unit for the second pixel value of the second pixel; a quantization width information generation unit for generating quantization width information having a quantization width used for quantization/decoding of the first and the second pixel value; and a quantization unit for generating a first and a second compressed encoded pixel value of n-bit length.

TECHNICAL FIELD

The present invention relates to a compression encoding and decoding ofdigital data. Specifically, the present invention relates to a deviceand a method for compressing and encoding a pixel signal outputted by animage pickup element, and for decoding the compressed and encoded pixelsignal.

BACKGROUND ART

The image pickup element which is mounted on a digital imaging apparatussuch as a digital camera and a cellular phone with a built-in cameraincludes such as a charge coupled type image pickup element (a CCD typeimage pickup element) and a metal oxide semiconductor type image pickupelement (a MOS type image pickup element). Further, the MOS type imagepickup element includes a complementary metal oxide semiconductor typeimage pickup element (a CMOS type image pickup element), an N-channelmetal oxide semiconductor type image pickup element (an NMOS type imagepickup element), and the like. In recent years, there is a tendency thatthe number of pixels of those image pickup elements has been increased,and each of them has been progressed as a high definition image pickupelement. The CCD type image pickup element has characteristics that itsdynamic range is wide with less noise, and the MOS type image pickupelement has characteristics that a simple structure is achieved becauseof using an MOS process with a single power source, which is suitablefor high resolution.

Next, a signal processing for imaging by using the digital imagingapparatus will be schematically explained.

An example of signal processing related to a still image pickup of oneframe, namely, a single image pickup, in the digital imaging apparatusis shown. Light from an object forms an image at a light receiving unitof the image pickup element, and a number of pixels arranged on thelight receiving unit accumulate charges of a certain quantity accordingto a light intensity. For one line of the light receiving unit,accumulated charges of pixels are read out pixel-by-pixel through anoutput unit of the image pickup element as analog signals, which arethen converted into digital pixel signals (RAW data) in ananalog/digital conversion unit, and is temporarily stored in a buffermemory such as a synchronous DRAM (SDRAM). When reading of the one line,A/D conversion, and writing into the SDRAM, etc, are completed,processing from the reading to the writing is similarly repeated for asecond line, a third line, . . . , and a final line, and data of oneframe of the image (one image pickup) is temporarily stored in theSDRAM. Next, a signal processing arithmetic operation such as a zoomprocessing of magnification/reduction is performed for the temporarilystored data, namely, RAW data, and the data after operation istemporarily stored in the SDRAM again. Next, by appropriately processingthe data by using a processor, the data is converted into the data(compressed data) of a compressed data format such as JPEG beingsuitable for storing the data, and thereafter is temporarily stored inthe SDRAM again. Then, the data is read out from the SDRAM at high speedby a direct memory access (DMA) control, etc., and the read data isstored in an external semi-permanent storage memory. Here, thesemi-permanent storage memory may be a storage medium generally used asan image recording medium of a digital camera such as an SD Memory Card.

By continuously executing the signal processing related to theaforementioned single image pickup, a continuous image pickup, namely, aconsecutive image pickup is realized. However, the processing from theRAW data to the data (compressed data) of the compressed data format isa time-consuming processing compared to the processing of reading outthe accumulated charges of the pixel and temporarily storing it as theRAW data. Therefore, in case of a continuous shooting, the processing ofstoring the RAW data in the SDRAM, and the processing of converting theRAW data into the compression data are performed simultaneously, and theread out RAW data is temporarily stored in the SDRAM additionally as faras a storage capacity of the SDRAM allows. Accordingly, in order toincrease the number of frames capable of continuous shooting, thestorage capacity of the SDRAM needs to be increased.

In addition, along with an increase of the number of pixels of the imagepickup element of recent years, a data size of the RAW data for oneframe of an image is also increased. Therefore, when the storagecapacity of the SDRAM is limited to a degree being same as that of aconventional product, the number of frames capable of continuousshooting is decreased along with the increase of the number of pixels ofthe image pickup element. Therefore, when a high definition of theimaging apparatus is realized by increasing the number of pixels of theimage pickup element, the storage capacity of the SDRAM simultaneouslyneeds to be made larger and the number of frames capable of continuousshooting needs to be secured. In addition, when the data size of the RAWdata becomes large, the SDRAM capable of being accessed at higher speedthan conventional is desired. However, a larger storage capacity of theSDRAM and a higher speed of access are disadvantageous in terms of itscost.

Conventionally, techniques for solving the aforementioned problem areproposed.

FIG. 11 is a block diagram illustrating a method for reducing a memoryusage of a frame memory being disclosed in Patent Document 1, with adigital still camera taken as an example.

First, a configuration of the digital still camera of the PatentDocument 1 will be explained. A digital still camera 100 includes: animage processing unit (CPU) 110; a flush memory 120; a JPEG-LSI 130; adisplay/capture controller 141; a buffer memory 142; a memory transfercontroller 143; an address bus switching unit 144; a read data latch145; an output level latch 146; a difference decompression/adder 147; adifference compression/decompression conversion table 148; asubtraction/difference compression unit 149; a write data latch 150; aninput level latch 151; a frame memory 160; an image output unit 170; aimage input unit 180; and a compression/decompression unit 140.

Next, an operation of the digital still camera 100 of the PatentDocument 1 will be explained. The data before compression inputted fromthe image input unit 180 of FIG. 11 is sent to thesubtraction/difference compression unit 149. Subtraction and differencecompression are performed therein. At that time, the compression isperformed with reference to the difference compression/decompressionconversion table 148. The data after compression being obtained as aresult is temporarily stored in the frame memory 160 through a data bus.Data before decompression being compressed that exists in the framememory 160 is converted into the data after decompression with referenceto the difference compression/decompression conversion table 148 in thedifference decompression/adder 147, and is outputted from the imageoutput unit 170.

This method needs to have the difference compression/decompressionconversion table 148 in the ROM, or the like, for performing thecompression of a difference value of the data. A circuit scale becomessmaller compared with a conventional method. Even so, it is inevitableto use the ROM, thus making a circuit structure larger, and a processingload is still large.

FIGS. 12 and 13 are schematic diagrams illustrating a method ofirreversible compression encoding of the digital signal disclosed inPatent Document 2. FIG. 12 is a block diagram of a single board CCD, andFIG. 13 is a flowchart of the irreversible compression encoding. FIG. 12shows a target pixel x, same color pixels f, e, d which are to beprocessed prior to the target pixel x, and different color adjacentpixels c, a, b being adjacent to the pixel x.

Next, the processing of a compression encoding method disclosed in thePatent Document 2 will be explained with reference to FIG. 13. Thismethod is a method of performing an entropy encoding to a predictederror between a predicted value y by an optimum prediction expressionand a value of a target pixel x in a single board CCD on which colorfilters R, G, B are arranged, so that the image data may be compressed.The method includes: calculating a predicted value by using the pixelvalue of a nearby pixel and the pixel value of the adjacent pixel onwhich a color filter of a color component different from that of thetarget pixel is arranged; calculating a predicted value by using thepixel value of the nearby pixel and a same color pixel on which thecolor filter of the same color as the target pixel is arranged;determining which of the predicted values is closer to the target pixelx; and deciding which a pixel value of the adjacent pixel or the samecolor pixel is to be used to calculate a predicted value of the nexttarget pixel based on this determination result.

In this method, quantization is performed with a nonlinear table, aconstant value is uniformly multiplied, and a table value to be used inan actual operation is calculated. Thus, compressibility can be changedin a irreversible conversion. The quantized data is furtherentropy-encoded.

In the method disclosed in the Patent Document 2, the memory such as aROM is also required in quantizing the predicted error Δ beingcalculated from a pixel value by using a predetermined quantizationtable, thus enlarging a circuit structure and increasing a processingload.

FIG. 14 is a block diagram of an image encoding device disclosed inPatent Document 3. FIG. 14 shows the image encoding device in which aninput pixel value of a dynamic range of d-bits is inputted from a pixelvalue input unit 101 and the input pixel value is encoded and convertedinto a quantized value of n-bits and the quantized value is thenoutputted from an output unit 105.

This image encoding device further includes: predicted value generatingmeans 106 for generating a predicted value for the input pixel value;linear quantizer generating means 102 for generating, in d-bit accuracy,a linear quantizer having a quantization width set at (d−n)-th power of2 and linear quantization representative points of the number which isobtained by subtracting an additional upper limit number beingpreliminarily set from n-th power of 2; and nonlinear quantizergenerating means 103 for generating the nonlinear quantizer having aquantization width in the vicinity of the predicted value being setsmaller than that of the linear quantizer by adding linear quantizationrepresentative points of the number which is less than the upper limitnumber to the linear quantizer in the vicinity of the predicted value.In the image encoding device, a quantization unit 104 quantizes theinput pixel value by using the nonlinear quantizer which is generated inthe nonlinear quantizer generating means 103 and a quantized value beingobtained is outputted.

The image encoding device disclosed in Patent Document 3 has no ROMtable for quantization. In this respect, this image encoding device isexpected to be realized as a smaller-scaled apparatus than the apparatusof the invention disclosed in Patent Documents 1 and 2.

Patent Document 1: JP 11-341288 A

Patent Document 2: JP 2000-244935 A

Patent Document 3: JP 10-056638 A

DISCLOSURE OF INVENTION Problem to be Solved by the Invention

The method described in Patent Document 1 needs to preliminarily preparea table for encoding/decoding, and hence a memory such as aencoding/decoding table buffer or a ROM is required for storing thetable. In addition, an apparatus for realizing the method of PatentDocument 1 has a large circuit structure and a load involved in theprocessing is also heavy.

At the same time, in the method described in Patent Document 2, one ofthe predicted values of which prediction error is smaller than that ofanother is selected after performing the predictions for the same colorpixel and for the different color pixel. Therefore, not only encodingbut also decoding needs to be performed to determine which predictionformula should be used. Also, an increase of a processing amount isinevitable. Further, the processing is a nonlinear processing using thetable, and therefore the circuit structure of the apparatus forrealizing this method becomes large and the load involved in theprocessing is also heavy.

In the apparatus for realizing the method described in Patent Document3, a circuit for calculating the predicted value needs to be configuredand the process for calculating the predicted value needs to beexecuted.

In view of the above-described problem, an object of the presentinvention is to provide a digital signal compression encoding/decodingdevice that can be mounted with a small scale circuit structure. Theprocess for encoding in the device is constructed with simplecalculations. Thus the device is capable of realizing highcompressibility with a low operation processing load. And the presentinvention provides a method for digital signal compression encoding anddecoding.

Means for Solving Problem

According to one aspect of the present invention, the present inventionprovides a digital data encoding device. The digital data encodingdevice receives pixel value data in a digital format indicating a signalfrom a light receiving unit in which at least a pixel for sensing afirst color and a pixel for sensing a second color are periodicallyarranged, and processes the data. The digital data encoding deviceincludes: a difference generation unit which outputs a difference valuebetween first pixel value data from a first pixel which senses the firstcolor and second pixel value data from a second pixel which senses thefirst color being positioned in the vicinity of the first pixel as afirst pixel difference value, and outputs a difference value of thirdpixel value data from a third pixel which senses a second color andfourth pixel value data from a fourth pixel which senses the secondcolor being positioned in the vicinity of the third pixel, as a secondpixel difference value; a quantization reference value determinationunit which obtains a maximum value between an absolute value of thefirst pixel difference value and an absolute value of the second pixeldifference value as a maximum pixel value difference, and determines avalue being greater than or equal to the obtained maximum pixel valuedifference as a quantization reference value; an offset value settingunit which sets a difference between the first pixel value data and thequantization reference value as a first offset value and sets adifference between the third pixel value data and the quantizationreference value as a second offset value; a value to be quantizedsetting unit which sets the difference between the second pixel valuedata and the first offset value as a first value to be quantized andsets the difference between the fourth pixel value data and the secondoffset value as a second value to be quantized; and a quantization unitwhich quantizes the first value to be quantized and the second value tobe quantized and obtains first compressed encoded pixel value data andsecond compressed encoded pixel value data.

In one aspect of the present invention, it is preferable that the firstcolor and the second color be different from each other.

In one aspect of the present invention, it is preferable that thedigital data encoding device further include an offset value zeroresetting unit which resets the offset value to zero when the offsetvalue being defined by the offset value setting unit is less than orequal to zero.

In one aspect of the present invention, it is preferable that thedigital data encoding device further include a quantization widthdetermination unit which determines a quantization width of thequantization.

In one aspect of the present invention, it is preferable that thequantization width be increased as the maximum pixel value differencebecomes larger.

In one aspect of the present invention, it is preferable that thedigital data encoding device further include a quantization widthinformation data generation unit which encodes the determinedquantization width into a code of m-bit length (m is a natural number).

In one aspect of the present invention, it is preferable that thequantization width determination unit determine the quantization widthto any one of preliminarily determined plural quantization widths.

In one aspect of the present invention, it is preferable that the numberof the plural quantization widths be less than or equal to m-th power of2.

In one aspect of the present invention, it is preferable that the m be2.

In one aspect of the present invention, it is preferable that thedigital data encoding device further include a compressed encoded imagedata generation unit, wherein the compressed encoded image datageneration unit generates compressed encoded image data of s-bit length(s is a natural number) including at least one of the quantization widthinformation data, the first compressed encoded pixel value data, and thesecond compressed encoded pixel value data and, more preferably, the sbe a multiple of 8.

In one aspect of the present invention, it is preferable that thecompressed encoded image data generation unit record the first pixelvalue data as they are in the compressed encoded image data as initialpixel value data.

In one aspect of the present invention, it is preferable that thequantization width determination unit obtain a maximum value between thenumbers of digits required for an unsigned integer binary value notationof each of the first pixel difference value and the second pixeldifference value so as to determine the quantization width from theplural quantization widths.

In one aspect of the present invention, it is preferable that thedigital data encoding device further include: an error correction unitwhich corrects the second pixel value data and generates error correctedpixel value data; and a digital data decoding unit which decodescompressed encoded pixel value data and outputs decoded pixel valuedata, wherein: the offset value setting unit sets the first offset valueby using the decoded pixel value data instead of the first pixel valuedata; and the value to be quantized setting unit sets the first value tobe quantized by using the error corrected pixel value data instead ofthe second pixel value data.

In one aspect of the present invention, it is preferable that thecorrection of the second pixel value data by the error correction unitbe performed by subtracting a correction value related to a differencebetween the first pixel value data and the decoded pixel value data fromthe second pixel value.

According to another aspect of the present invention, the presentinvention provides a digital data decoding device. The digital datadecoding device includes: a compressed encoded image data input unitwhich inputs compressed encoded image data of s-bit lengths (s is anatural number) which has an initial pixel value data part in whichfirst pixel value data of a first pixel which senses a first color arerecorded as they are as first initial pixel value data and a compressedencoded pixel value data part in which first compressed encoded pixelvalue data indicating second pixel value data of a second pixel whichsenses the first color being positioned in the vicinity of the firstpixel are recorded; an offset value setting unit which obtains adifference between the first initial pixel value data and a firstquantization reference value being set as a first offset value; aninverse quantization unit which inversely quantizes the first compressedencoded pixel value data by using a first quantization width being setto obtain first inversely quantized pixel value data; and a decodedpixel value generation unit which obtains a sum of the first inverselyquantized pixel value data and the first offset value to generate firstdecoded pixel value data.

In one aspect of the present invention, it is preferable that thirdpixel value data of a third pixel which senses a second color beingpositioned nearby the first pixel be further recorded as they are assecond initial pixel value data in the initial pixel value data part. Itis also preferable that second compressed encoded pixel value data whichindicates fourth pixel value data of a fourth pixel which senses thesecond color being positioned in the vicinity of the third pixel befurther recorded in the compressed encoded pixel value data part. It isfurther preferable that: the offset value setting unit obtain adifference between the second initial pixel value data and the firstquantization reference value as a second offset value; the inversequantization unit further inversely quantize the second compressedencoded pixel value data by using the first quantization width so as toobtain second inversely quantized pixel value data; and the decodedpixel value generation unit further obtain a sum of the second inverselyquantized pixel value data and the second offset value to generatesecond decoded pixel value data.

In one aspect of the present invention, it is preferable that the firstcolor and the second color be different from each other.

In one aspect of the present invention, it is preferable that: thirdcompressed encoded pixel value data indicating fifth pixel value data ofa fifth pixel which senses the first color being positioned in thevicinity of the second pixel and being nearby the fourth pixel befurther recorded in the compressed encoded pixel value data part; theoffset value setting unit further obtain a difference between the firstdecoded pixel value data and second quantization reference value beingset as a third offset value; the inverse quantization unit furtherinversely quantize third compressed encoded pixel value data by usingthe second quantization width being set to obtain third inverselyquantized pixel value data; and the decoded pixel value generation unitfurther obtain a sum of the third inversely quantized pixel value dataand the third offset value to generate third decoded pixel value data.

In one aspect of the present invention, it is preferable that thedigital data decoding device further include an offset value zeroresetting unit which resets an offset value to zero when the offsetvalue being defined by the offset value setting unit is less than orequal to zero.

In one aspect of the present invention, it is preferable that thecompressed encoded image data include a quantization width informationdata part in which at least one of first quantization width informationdata having information regarding the first quantization width andsecond quantization width information data having information regardingthe second quantization width be recorded.

In one aspect of the present invention, it is preferable that thedigital data decoding device further include a quantization widthsetting unit which sets a quantization width of the inverse quantizationto any one of the preliminarily determined plural quantization widths,wherein the quantization width setting unit sets the first quantizationwidth and the second quantization width to ones of the pluralquantization widths, respectively, on the basis of the firstquantization width information data and the second quantization widthinformation data.

In one aspect of the present invention, it is preferable that thedigital data decoding device further include a quantization referencevalue setting unit which sets a quantization reference value of theinverse quantization to any one of the preliminarily determined pluralquantization reference values, wherein the quantization reference valuesetting unit sets the first quantization reference value and the secondquantization reference value, respectively, on the basis of the firstquantization width information data and the second quantization widthinformation data.

In one aspect of the present invention, it is preferable that the firstquantization width information data and the second quantization widthinformation data be pieces of data of m-bit lengths (m is a naturalnumber), respectively.

In one aspect of the present invention, it is the plural quantizationwidths and the plural quantization reference values be less than orequal to m-th power of 2, respectively.

In one aspect of the present invention, it is preferable that the m be2.

According to further aspect of the present invention, the presentinvention provides a digital data encoding method. The digital dataencoding method is a method for receiving pixel value data in a digitalformat indicating a signal from a light receiving unit in which pixelsfor sensing a first color and pixels for sensing a second color areperiodically arranged and processing the data. The digital data encodingmethod includes: generating and outputting a difference value betweenfirst pixel value data of a first pixel which senses a first color andsecond pixel value data of a second pixel which senses the first colorand is positioned in the vicinity of the first pixel as a first pixeldifference value and generating and outputting a difference valuebetween third pixel value data of a third pixel which senses a secondcolor and fourth pixel value data of a fourth pixel which senses asecond color and is positioned in the vicinity of the third pixel as asecond pixel difference value; obtaining a maximum value between anabsolute value of the first pixel difference value and an absolute valueof the second pixel difference value as a maximum pixel valuedifference, determining a value being greater than or equal to themaximum pixel value difference, and determining the value as aquantization reference value; setting a difference between the firstpixel value data and the quantization reference value as a first offsetvalue and setting a difference between the third pixel value data andthe quantization reference value as a second offset value; setting adifference between the second pixel value data and the first offsetvalue as a first value to be quantized and setting a difference betweenthe fourth pixel value data and the second offset value as a secondvalue to be quantized; and quantizing the first value to be quantizedand the second value to be quantized to obtain first compressed encodedpixel value data and second compressed encoded pixel value data,respectively.

According to yet further aspect of the present invention, the presentinvention provides a digital data decoding method. The digital datadecoding method includes: inputting compressed encoded image data ofs-bit lengths (s is a natural number) which has an initial pixel valuedata part in which first pixel value data of a first pixel which sensesa first color are recorded as they are as first initial pixel value dataand a compressed encoded pixel value data part in which first compressedencoded pixel value data indicating second pixel value data of a secondpixel which senses the first color being positioned in the vicinity ofthe first pixel are recorded; obtaining a difference between the firstinitial pixel value data and first quantization reference value beingset as a first offset value; inversely quantizing the first compressedencoded pixel value data by using first quantization width being set toobtain first inversely quantized pixel value data; and obtaining a sumof the first inversely quantized pixel value data and the first offsetvalue to generate first decoded pixel value data.

Effects of the Invention

The digital signal compression encoding and decoding device and themethod of the present invention has no need to provide a table forencoding and decoding, and, hence, is constituted of a small scalecircuit structure, and is capable of realizing high data compressibilityby using a relatively simple operational processing.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of a digital still camera according to thepresent invention.

FIG. 2 is a schematic diagram of usage of a storage area of an SDRAM.

FIG. 3 is a schematic diagram of an arrangement of pixels of a lightreceiving unit of an image pickup element.

FIG. 4 is a block diagram of a CODEC 13 according to a first embodimentof the present invention.

FIG. 5A is a flowchart of encoding processing.

FIG. 5B is a flowchart of encoding processing.

FIG. 6 is a graph showing a relation of quantities in quantization.

FIG. 7 is a graph illustrating a detail of quantization processing witha pixel value g2 taken as an example.

FIG. 8A is a flowchart of decoding processing.

FIG. 8B is a flowchart of decoding processing.

FIG. 9 is a block diagram of a CODEC 113 according to a secondembodiment.

FIG. 10 is a block diagram of a CODEC 213 according to a thirdembodiment.

FIG. 11 is a block diagram of a configuration of a digital still cameradisclosed in Patent Document 1.

FIG. 12 is a partial schematic view of a single board CCD.

FIG. 13 is a flowchart of a digital signal compression encoding methoddisclosed in Patent Document 2.

FIG. 14 is a block diagram of an image encoding device disclosed inPatent Document 3.

EXPLANATIONS OF LETTERS AND NUMERALS

-   1 digital still camera (DSC)-   3 lens-   5 image pickup element-   7 image pickup element drive unit-   9 Signal pre-processing unit-   11 Analog/digital conversion unit (ADC)-   13 CODEC-   15 SDRAM-   17 YC processing unit-   19 JPEG processing unit-   21 controller-   23 external interface-   25 SD Memory Card-   27 display unit-   29 pixel-   31 encoding unit-   33 decoding unit-   35 target pixel value input unit-   37 37 difference generation unit-   39 pixel value storage unit-   41 difference quantization range determination unit-   43 zone quantization width determination unit-   45 initial pixel value generation unit-   47 47 quantization unit-   47 a quantization reference value determination unit-   47 b offset value setting unit-   47 c value to be quantized setting unit-   47 d quantization unit-   47 e offset value zero resetting unit-   49 class value code generation unit-   51 packing unit-   51 a compressed encoded image data generation unit-   51 b compressed encoded image data output unit-   53 unpacking unit-   55 inverse quantization processing unit-   55 a offset value setting unit-   55 b inverse quantization unit-   55 c decoded pixel value generation unit-   55 d offset value zero resetting unit-   55 e quantization width setting unit-   55 f quantization reference value setting unit-   57 output unit-   59 error detection unit-   61 coefficient multiplication unit-   63 integer conversion unit-   65 error correction unit-   67 difference quantization range extraction unit-   69 distribution ratio analysis unit

BEST MODE FOR CARRYING OUT THE INVENTION Embodiment 1

<Configuration of a Digital Still Camera>

FIG. 1 is a block diagram of a digital still camera (DSC) 1 on which adigital signal compression encoding/decoding device (CODEC) 13 accordingto the present invention is mounted. Light incident to a lens 3 from anobject being not shown is collected by the lens 3, and an image thereofis formed in a light receiving unit being not shown of an image pickupelement 5. The image pickup element 5 is a CCD type imaging element. Apixel being not shown of the image pickup element 5 accumulates chargesaccording to amount of the incident light. An image pickup element driveunit 7 outputs the accumulated charges as an analog pixel signal at apredetermined timing, and the signal is sent to a signal pre-processingunit 9. The signal pre-processing unit 9 applies pre-processing to theanalog pixel signal, and sends the analog pixel signal to ananalog/digital conversion unit 11. The analog/digital conversion unit(ADC) 11 converts the analog pixel signal into a digital pixel signaland outputs pixel value data in a digital format.

In the digital still camera 1 as a imaging apparatus, the digital pixelsignal, which is the pixel value data of the digital format, beingoutputted from the

ADC 11 is inputted to the digital signal compression encoding/decodingdevice (CODEC) 13. The CODEC 13 analyses the inputted digital pixelsignal and sends compressed encoded pixel value data being obtained byanalyzing the inputted digital pixel signal and compression-encoding thedigital pixel signal, and information regarding compression encoding,etc., to an SDRAM 15 being as a buffer memory. A group of data thusbeing sent to the SDRAM 15 constitutes compressed encoded image data.Compression encoding and decoding in the CODEC 13 will be explained indetail below. The compressed encoded pixel value data etc. beingoutputted is stored in the SDRAM 15. The compressed encoded pixel valuedata etc. being stored in the SDRAM 15 is sent back to the CODEC 13again and decoded therein to generate decoded pixel value data. Thedecoded pixel value data is sent to a YC processing unit 17 and isconverted into luminance and color difference data. The luminance andcolor difference data is sent back to the SDRAM 15 again and is storedtherein. The luminance and color difference data being stored in theSDRAM 15 is sent to a JPEG processing unit 19 and is subjected to JPEGcompression processing. Thus obtained image data (JPEG image data),which is turned into JPEG data, is stored in the SDRAM 15. The JPEGimage data is sent to an SD Memory Card 25 by DMA control etc. at highspeed, which is an external storage medium, and is stored therein. A CPUbeing not shown and being included in a controller 21 executes a programstored in a memory being not shown and being included in the controller21 and controls the aforementioned processing. In addition, thecontroller 21 is also capable of sending information stored in the SDRAM15 via an external interface 23 to a display unit 27 and the SD MemoryCard 25. Further, the controller 21 is also possible to read data ofinformation stored in the SD Memory card 25.

Note that the image pickup element 5 may also be a MOS type image pickupelement. The image pickup element is not limited to the image pickupelement having the pixels which sense three colors such as a pixel whichsenses red color, a pixel which senses green color, and a pixel whichsenses blue color. Also, the image pickup element 5 may be the imagepickup element having pixels of the complementary color system. Inaddition, the image pickup element 5 may be the image pickup elementhaving only pixels which sense two colors being different from eachother. Further, the image pickup element 5 may be the image pickupelement having pixels which sense only one color, or may be the imagepickup element having not the pixels which sense light of a particularwavelength band only but the pixels of a uniform characteristic whichsense light of wide range. The external storage medium is not limited tothe SD Memory Card 25 but may be the one generally used as the imagestorage medium of a digital camera.

FIG. 2 is a diagram schematically showing a usage of the storage area ofthe SDRAM 15. As described above, the SDRAM 15 stores three kinds ofdata, namely, compressed encoded image data, luminance and colordifference data, and JPEG image data. In this description, the areastoring the compressed encoded image data is called as a storage area 15a, the area storing the luminance and color difference data is called asa storage area 15 b, and the area storing the JPEG image data is calledas a storage area 15 c. A boundary of the storage area (broken line)shown in this figure is only a ritual one and does not show its actualratio of usage.

In the imaging apparatus 1 according to the present invention, in theimage pickup processing, first, a digital pixel signal (RAW data) whichis based on the signal value outputted from the image pickup element 5is compression-encoded by the CODEC 13 and the data is stored in thestorage area 15 a as compressed encoded pixel value data (a part of adata group constituting the compressed encoded image data). Next, afterbeing subjected to decoding processing in the CODEC 13, the compressedencoded pixel value data is sent to the YC processing unit 17, and isthen subjected to data processing and stored in the storage area 15 b asthe luminance and color difference data. Then, the luminance and colordifference data is subjected to JPEG compression processing in the JPEGprocessing unit 19, and is stored in the storage area 15 c as the JPEGimage data. Thereafter, the JPEG image data is transferred to the SDMemory Card 25. Generation processing of the JPEG image data is theprocessing requiring much time with compared to other processing.Therefore, when continuous image pickup such as continuous shooting isexecuted, data that waits for the JPEG processing is increased, thus thestorage area of the SDRAM 15 is encumbered. Therefore, in order toeffectively use the storage area of the SDRAM 15 as much as possible andincrease the number of shoots that can be continuously shot, it iseffective to reduce a usage of the storage area 15 a. The digital signalcompression encoding according to the present invention has an advantageof reducing the amount of use of the area 15 a since the digital pixelsignal (RAW data) is compression-encoded and sent to the SDRAM 15. Thus,the storage area that may be used for the areas 15 b and 15 c can betaken large. As a result, the number of shoots of the digital stillcamera 1 which can be consecutively taken can be increased. In addition,the amount of data flow from/to the CODEC 13 to/from the SDRAM 15 isreduced, and therefore an advantage of shortening a processing time andreducing a consumption power also can be obtained.

Next, explanation will be given to the processing in the CODEC 13, whichis a encoding and decoding device of the digital still camera, namely,the imaging apparatus of the present invention.

Before explanation is given to a configuration of the CODEC 13, first,the digital pixel signal (RAW data) which is inputted to the CODEC 13will be explained with reference to FIG. 3. FIG. 3 is a diagram of anarrangement of pixels 29 in a light receiving unit of the image pickupelement 5. A plurality of pixels 29 are arranged in the light receivingunit and a color filter of any one of red color (R), green color (G), orblue color (B) is disposed on each pixel 29. A wavelength band, (whichis color for a visible light region), being sensed by each pixel 29 islimited by the color filter. The pixels that sense different colors(different wavelength bands) are periodically disposed in the lightreceiving unit. The style of the arrangement of the three color filtersis a so-called Bayer array. An image pickup element drive unit (seeFIG. 1) outputs charges accumulated in the pixel 29 sequentially fromthe pixel of a left side for each one line. For example, at first,accumulated charges are read out from a pixel G1 of a left end of afirst line L1 sequentially in an order of G1, R1, G2, R2, . . . , andnext, the accumulated charges are read out from a pixel B1 of a left endof a second line L2 sequentially in an order of B1, G1, B2, G2, . . . .Read charges are subjected to pre-processing in the pre-processing unit9, and are converted into the digital pixel signal in an ADC 11. In theADC 11, a signal from each pixel is converted into the digital pixelsignal of 12 bits. Therefore, the digital pixel signals (pieces of pixelvalue data) having 12 bit length respectively are inputted to the CODEC13, from the pixel G1 of the left end of the first line L1 in an orderof pixel G1, pixel R1, pixel G2, pixel R2, . . . . When an input of thedigital pixel signal from the pixel included in the first line L1 iscompleted, the digital pixel signal from the pixel included in thesecond line L2 is inputted to the CODEC 13 similarly.

<Configuration of CODEC 13>

FIG. 4 is a block diagram of the CODEC 13 according to this embodiment.The configuration of the CODEC 13 will be schematically explained belowalong with a flow of an encoded signal and a decoded signal. Anillustration of an operation of the CODEC 13 will be described in detaillater by using a certain value as a digital pixel signal, namely, apixel value as an example.

Referring to FIG. 4, the CODEC 13 is divided broadly into an encodingunit 31 and a decoding unit 33. The encoding unit 31 which constitutesthe digital data encoding device can input an output from the ADC 11 andoutput the data to the SDRAM 15 and the decoding unit 33. The decodingunit 33 which constitutes the digital data decoding device can input theoutput from the encoding unit and the SDRAM 15 and output the data tothe encoding unit 31 and the YC processing unit 17.

<Compression Encoding Processing in the Encoding Unit 31>

Explanation will be given hereunder regarding the processing for thecompression encoding of pixel value data (a digital pixel signal value)in the encoding unit 31 of the CODEC 13 with reference to FIGS. 4 and5A. FIG. 5A is a flowchart of the compression encoding processing.

The pixel value of each pixel, which is outputted from the ADC 11, isinputted to a target pixel value input unit 35 at a predeterminedtiming. In this embodiment, each digital pixel signal value (pixel valuedata) is digital data of 12 bit length. That is, an input pixel valuedata bit length d is 12 in this embodiment.

According to FIG. 4, the pixel value data being inputted to the inputunit 35 is sent to an initial pixel value generation unit 45, adifference generation unit 37, a quantization unit 47, and a pixel valuestorage unit 39 as a target pixel value.

The initial pixel value generation unit 45 sends the target pixel valuehaving received to a packing unit 51 as initial pixel value data ofd(=12) bit length. The packing unit 51 has a compressed encoded imagedata generation unit 51 a which performs a generation of the compressedencoded image data and a compressed encoded image data output unit 51 bwhich performs the output processing. When the generation unit 51 a ofthe packing unit 51 determines that the initial pixel value data havingreceived should be recorded in the compressed encoded image data, thegeneration unit 51 a of the packing unit 51 performs processing whereinthis initial pixel value data is included in the compressed encodedimage data. When the generation unit 51 a determines that the initialpixel value data having received may not be recorded in the compressedencoded image data as an initial pixel value data, the generation unit51 a ignores the initial pixel value having received. At least oneinitial pixel value for each pixel colors (R, G, B) should be present inthe image data of one frame. However, in this embodiment, regarding thepixel value of each color, at least one initial pixel value data isrecorded for every packing data of s-bit length, which will be describedlater. Regarding a certain packing data, the initial pixel value data ofa certain color needs not be recorded if a pixel value of the certaincolor not recorded at all in the certain packing data. Here, the initialpixel value generation unit 45 does not substantially perform anyparticular processing, which outputs data being inputted as it is. Theinitial pixel value generation unit 45 is shown in the figure for thepurpose of clarifying the processing of inputting the target pixel valuein the packing unit 51 as the initial pixel value data. (Steps S101 andS102 of FIG. 5A).

The pixel value storage unit 39 temporarily stores a plurality of piecesof pixel value data, and output them to the difference generation unit37 and the quantization unit 47 at an appropriate timing. The pixelvalues being stored are at least one of a previous target pixel valuebeing previously inputted to the CODEC 13 as the pixel value data priorto the present target pixel value and the pixel value which ispreviously compressed and encoded, sent to the decoding unit 33, andsubjected to the processing of decoding so as to be decoded, namelydecoded pixel value data. The storage unit 39, out of the storedplurality of pieces of the pixel value data, sends a pixel value data ora decoded pixel value data of the pixel as a left side nearby same colorpixel value to the difference generation unit 37 at a predeterminedtiming (see FIG. 3). The pixel 29 is, in the light receiving unit of theimage pickup element, positioned in the vicinity of the pixel being asource of the present target pixel value. In addition, the pixel 29 hasthe same color as that of the pixel of the present target pixel value.The decoded pixel value is a data being subjected to the compressionencoding processing and further decoded. In the light receiving unit,the pixel of the left side nearby same color pixel value is usuallypresent in the left side of the pixel shown as the present target pixelvalue. When no pixel having the same color is present in the left sideof the same line, the pixel value of the same colored pixel on the upperline which is present nearby may be used as the left side nearby samecolor pixel value. When no stored pixel value is present, apredetermined value may be outputted as the left side nearby same colorpixel value. Note that, in this embodiment, the same color pixel beingnearest to the target pixel is used as the same color pixel beingpositioned nearby. However, a pixel value from a pixel can be used indifference generation as long as the pixel is positioned nearby thetarget pixel.

The difference generation unit 37 generates a difference between thetarget pixel value being sent from the input unit 35 and the left sidenearby same color pixel value being sent from the storage unit 39 (=thetarget pixel value—the left side nearby same color pixel value). (StepS103 of FIG. 5A). However, when the present target pixel value isrecorded as the initial pixel value, the processing for the presenttarget pixel value in the difference generation unit 37 is unnecessary.The value of the difference being generated is sent to a differencequantization range determination unit 41 as a pixel difference value.

The difference quantization range determination unit 41 obtains, basedon an absolute value of a difference value regarding each target pixelvalue, namely, an absolute differential value which is sent from thedifference generation unit 37, a “quantization range” of an absolutedifferential value of each target pixel value. The “quantization range”represents the number of digits of the absolute differential value inbinary form, namely, the (a binary digit notation) absolute differentialvalue. Specifically, the “quantization range” means the number of digits(the number of bits) required in a signed or an unsigned integer binarynotation of the absolute differential value, namely, an unsigned integerbinary notation of the difference value. The quantization range is sentto a zone quantization width determination unit 43. (Step S104 of FIG.5A.)

The zone quantization width determination unit 43 starts followingprocessing after receiving quantization ranges of other pixels beingincluded in the same “zone,” which will be described later, namely,quantization ranges of the second pixel, third pixel, fourth pixel, andthe like from the difference quantization range determination unit 41.(Step S105 of FIG. 5A). The term “zone” here indicates a set of pixelsof predetermined number, which are mutually positioned nearby, (and aset of the pixel values of those pixels). The “nearby” pixels indicatethe adjacent or nearest pixel with respect to a certain pixel. Everypixel being included in the zone may be adjacent to any one of otherpixels included in the same zone. The CODEC 13 of the present inventionquantizes the pixel values included in the same zone according to thesame quantization width, namely, “zone quantization width” as will bedescribed later with same quantization accuracy (same quantizationrepresentative value interval). The number of pixels included in onezone may be 4 (p=4) in this embodiment. However, the number of pixelsincluded in one zone, p, is not limited to 4, and may be the integerfrom 1 to the total number of pixels included in the light receptionelement. In addition, in compressing and encoding the pixel values ofone frame of the image, the number p which is the number of pixels beingincluded in the zone may be variable. The pixel which outputs the pixelvalue data used as the initial pixel value may not be included in anyzone.

The zone quantization width determination unit 43 is a block fordetermining the quantization width. The zone quantization widthdetermination unit 43 determines a zone quantization width, which isinformation regarding a quantization coefficient or the quantizationwidth for quantizing the target pixel value. (Step S106 of FIG. 5A)

It may also be described that the “zone quantization width” is the datafor transferring the information regarding a width (interval) of thequantization representative value for quantizing the pixel values beingincluded in the same zone. The zone quantization width may be an integervalue being greater than or equal to zero. The zone quantization widthis equal to a difference between: a value obtained by adding 1 to thequantization range which corresponds to a maximum pixel value differencewhich is a maximum value of the difference values being differencesbetween the pixel values included in the zone and the left side nearbysame color pixel value; and the bit number, n, of the data obtained bycompressing and encoding the pixel value data, namely, “compressed andencoded pixel value data bit number (n).” This “compressed and codedpixel value data bit number (n)” may be a predetermined value, and inthis embodiment, this value is n=8. This means that the compressedencoded pixel value data corresponding to its pixel value is recorded asthe data having an 8 bit length, meanwhile the data of the target pixelvalue being inputted has 12 bit length. However, the bit number n ofthis compressed encoded pixel value data is not limited to eight. Inaddition, in a series of compression encoding for the pixel values ofone frame of the image, this value may be variable. Further, as a resultof the aforementioned calculation, when the zone quantization width is anegative number, the zone quantization width may be reset to 0.

Here, it can be also described that the “quantization representativevalue interval” indicates the number of integer values beforequantization which are to be quantized to an identical quantized valueby the quantization of those integer values. For example, when norounding is performed at all in quantizing the digital data expressing acertain integer value, the quantization representative value interval is0-th power of 2=1. In this case, the quantization is performed withone-to-one correspondence between the integer value and its quantizedvalue being kept. In addition, when the quantization representativevalue interval is expressed as 1st power of 2=2, this means that the 1bit of a lowest-order of digital data of an integer value is rounded atthe quantization. Consequently, two kinds of digital data each of whichindicate two different kinds of integer values are quantized to anidentical quantized value. Further, when the quantization representativevalue interval is expressed as 2nd power of 2=4, this means that the 2bits of a lowest-order of digital data of an integer value is rounded atthe quantization. Consequently, four kinds of digital data each of whichindicate four different kinds of integer values are quantized to anidentical quantized value. The same goes for a case of rounding 3 bitsor more of lowest-order of digital data.

The zone quantization width thus determined is sent to the quantizationunit 47 and a class value code generation unit 49.

The class value code generation unit 49 generates quantization widthinformation data. The class value code generation unit 49 sends a “classvalue” of m bits, namely, quantization width information data, whichcorresponds to the zone quantization width having received, to thepacking unit 51. In this embodiment, m=2. This “class value” is the dataindicating a quantization coefficient in the quantization of the pixelvalue data included in the zone (bit numbers being rounded byquantization), which is then processed and recorded by the packing unit51 together with the compressed encoded pixel value data which is thecompressed pixel value data. (Step S107 of FIG. 5A)

In FIG. 4 again, the quantization unit 47 performs quantization based onthe target pixel value, left side nearby same color pixel value, andzone quantization width having received, and sends a result to thepacking unit 51 as the compressed encoded pixel value data, which is thecompressed value of the target pixel value. (Step S108 of FIG. 5A)

The quantization unit 47 includes:

a quantization reference value determination unit 47 a which determinesa quantization reference value;

an offset value setting unit 47 b which sets an offset value or anoffset candidate value from the pixel value data and the quantizationreference value;

an offset value zero resetting unit 47 e which resets the offset(candidate) value being zero or less to the offset value having a valuezero;

a value to be quantized setting unit 47 c which sets a value to bequantized which is a value actually quantized; and

a quantization unit 47 d which quantizes the value to be quantized.

The processing by the quantization unit 47 is described in detail byusing an processing example described below.

The packing unit 51 has a function of collecting several kinds ofinputted data, namely, pieces of compressed encoded data and packingthem into a piece of an appropriate size (generation unit 51 a) and afunction of outputting the packed data to the SDRAM 15 and an unpackingunit 53 (output unit 51 b). Also, the packing unit 51 can output eachdata as it is without packing. In this embodiment, a size of the packingdata, which is the compressed encoded image data, is s-bits. The s (s:natural number) may be an integral multiple of 8 (such as 8, 16, . . . ,64, . . . , 96, etc.). When a small number of unused bits which can notbe used for recording meaningful data remains, a predetermined dummydata may be recorded therein.

<Decoding Processing by the Decoding Unit 33>

The unpacking unit 53 is the block which inputs the compressed encodedimage data. The unpacking unit 53 analyzes packing data or unpacked datahaving received from the packing unit 51 and packing data havingreceived from the SDRAM 15. The unpacking unit 53 separates the packingdata into: an initial pixel value data part including initial pixelvalue data; a quantization width information data part including theclass value; and a compressed encoded pixel value data part includingthe compressed encoded pixel value data, and further separates thosedata parts into one or more pieces of data. Those pieces of data aresent to an inverse quantization unit 55.

The inverse quantization unit 55 includes:

an offset value setting unit 55 a which sets an offset value by usingthe initial pixel value data or the decoded pixel value data, and thequantization reference value;

an inverse quantization unit 55 b which inversely quantizes thecompressed encoded pixel value data to obtain inversely quantized pixelvalue data;

a decoded pixel value generation unit 55 c which generates the decodedpixel value data by using the pixel value data or the decoded pixelvalue data, and the inversely quantized pixel value data;

an offset value zero resetting unit 55 d which resets an offset value oran offset candidate value being zero or less to zero;

a quantization width setting unit 55 e which sets a quantization width;and

a quantization reference value setting unit 55 f which sets aquantization reference value.

The inverse quantization unit 55 inversely quantizes the compressedencoded pixel value data using several kinds of pieces of data havingreceived. The inverse quantization unit 55 performs reversed processingwith respect to the processing in the quantization unit 47 to obtain aninversely quantized pixel value data. The inverse quantization unit 55further processes the inversely quantized pixel value data to obtain thedecoded pixel value data, which is decoded data. The decoded pixel valuedata (12-bit length data in this embodiment) is sent to the output unit57. As to the initial pixel value data, it is sent to the output unit 57as it is since the initial pixel value data is received asnon-compressed data (actual data) of 12-bit length.

The output unit 57 sends initial pixel value data and decoded pixelvalue data having received to the YC processing unit 17 and the imagepixel storage unit 39. In the pixel value storage unit 39, the decodedpixel value data having received can be utilized for differencegeneration processing and quantization processing on other target pixelvalues.

<Exemplified Compression Encoding Processing>

The compression encoding processing and the decoding processing by theCODEC 13 according to the present embodiment will be explained in detailwith reference to FIG. 5B, FIG. 6, and FIG. 7. Specific numerical valuesare given to the target pixel value, etc. FIG. 5B is a flowchart ofdetails of the processing in step S108 of FIG. 5A.

12-bit data is inputted from the ADC 11 to the target pixel value inputunit 35 at a predetermined timing. At first, a pixel value g1 of a pixelG1 is inputted, and next, a pixel value r1 of a pixel R1 is inputted,and similarly g2, r2, g3, r3, . . . are inputted. In one image taking,preferably, the pixel value which is inputted first with respect to eachcolor (g1 and r1 in this example) is sent to the initial pixel valuegeneration unit 45 and is processed as the initial pixel value data, andthen, sent to the packing unit 51. The initial pixel value is recordedas the actual data. The initial pixel value is not subjected to thedifference generation processing and the quantization processing.However, the initial pixel value is necessary for applying thecompression encoding processing to the pixel value data of the next samecolor pixel. Therefore, the initial pixel value is sent to thequantization unit 47 and the pixel value storage unit 39 and istemporarily stored therein. In addition, if the data of the target pixelvalue is processed as the initial pixel value data at an interval ofpredetermined number of pixels, an effect of resetting (removing) anaccumulated error due to quantization is expected and so that animprovement in SN ratio of a reproduced image with respect to aninputted image is expected.

In the present embodiment, four pixels are included in one zone (namely,p=4). The pixel values g1 and r1 each of which are respectively treatedas the initial pixel value data are sent to the packing unit 51 as theyare. And then, they are respectively recorded into the packing data aspieces of data of 12-bit width. Next, pixels G2, R2, G3, and R3 aretreated as the pixels all of which are included in a first zone. Thesame goes for a second zone and a third zones.

Next, the processing on the pixel values g2, r2, g3, and r3, all ofwhich are included in the first zone, will be explained. For the pixelvalues g1, r1, g2, r2, g3, and r3, values described in a table below(table 1) are used as a specific example of the values.

TABLE 1 Pixel G1 R1 G2 Pixel value g1 r1 g2 Pixel value 2000 250 2100(decimal notation) Pixel value 011111010000 000011111010 100000110100(binary notation) Pixel R2 G3 R3 Pixel value r2 g3 r3 Pixel value 501700 100 (decimal notation) Pixel value 000000110010 001110100100000001100100 (binary notation)

Pixel value g2 is designated as a target pixel value. The pixel value g2is sent to the quantization unit 47, the difference generation unit 37,and the pixel value storage unit 39 as the target pixel value. At thistime, the pixel value g1 is sent from the pixel value storage unit 39 tothe difference generation unit 37 as the left side nearby same colorpixel value to obtain the difference value between those pixel values.That is, if Δg2 is set as the difference value at this time,

Δg2=g2−g1.

The difference value Δg2 is sent to the difference quantization widthdetermination unit 41.

When the generalization of this difference generation processing isattempted, it can be expressed as Δci=ci−c(i−1), (c represents the colorof the pixel, and c:r, g, or b, and i is the integer value indicating anorder of the pixel). ci is the target pixel value and c(i−1) is the leftside nearby same color pixel value. The left side nearby same colorpixel value is the target pixel value having previously inputted or thedecoded pixel value data which had been subjected to compressionencoding processing once and has been decoded. As is the case where i=1,when there is no pixel value data designated by a subscript “i−1”, thepixel value data of the pixel being positioned in the light receivingunit of the image pickup element and above and vicinity of the targetpixel may be used. In this example, the target pixel value having beenpreviously inputted is used as the left side nearby same color pixelvalue. In generating the difference for pixel values which are includedin a second or later zone, the decoded pixel value data can be used asthe left side nearby same color pixel value.

Similarly, the difference value Δg2 regarding the r2 as the target pixelvalue, the difference value Δg3 regarding the g3 as the target pixelvalue, and the difference value Δg3 regarding the r3 as the target pixelvalue are obtained. As to the target pixel value g3, the differencebetween the target pixel value g3 and the pixel value g2 being stored inthe pixel value storage unit 39 is the difference value Δg3.

The difference quantization width determination unit 41 obtains anabsolute value of each difference value having received, namely, theabsolute differential value, and obtains the number of digits of thebinary notation of the absolute differential value, namely, thequantization range (a number of bits required, bit( )).

As to Δg2, the difference value is +100, and therefore, the absolutedifferential value is 100 as it is. The 100 of decimal notation is1100100 in binary notation. Therefore, the required bit number,bit(|Δg2|) is 7. In this description, decimal X is noted as &D(X), andbinary Y is noted as &B(Y). Similarly obtained four quantization ranges(bit(|Δg2|), bit(|Δr2|), bit(|Δg3|), and bit(|Δr3|) are sent to the zonequantization width determination unit 43.

When the generalization of the processing in the difference quantizationwidth determination unit 41 is attempted, it can be expressed that theabsolute value of the difference value, namely, the absolutedifferential value is obtained and the number of digits of the absolutedifferential value in binary notation is outputted to the zonequantization width determination unit 43 as the quantization range.

After receiving an input of each quantization range regarding the pixelvalue of the pixel being included in the first zone, the zonequantization width determination unit 43 determines the zonequantization width.

The zone quantization width is equal to the difference between the valueobtained by adding 1 to the maximum value among the quantization rangesof the pixels being included in the zone, and the number of bits of thecompressed encoded pixel value data. However, in case where the zonequantization width is a negative number, the zone quantization width isset to zero. That is, the zone quantization width is determined on thebasis of the maximum pixel value difference which is the maximum valueamong the absolute differential values between the left side nearby samecolor pixel value and the pixel value being included in this zone.

In this embodiment, the pixel value data is compressed and encoded tothe compressed encoded pixel value data of 8-bit length, and thus, thebit number of the compressed encoded pixel value data n=8. Therefore,the zone quantization width is 2(=9+1−8).

When the zone quantization width determination unit 43 determines thezone quantization width, the zone quantization width determination unit43 sends the zone quantization width to the quantization unit 47 and theclass value code generation unit 49.

In the quantization unit 47, a “value to be quantized”, which is a valuebased on the received difference value between the target pixel valueand the left side nearby same color pixel value, is quantized based onthe zone quantization width.

The zone quantization width of the first zone is 2. There is acorrelation between the zone quantization width and an accuracy of thequantization executed by the quantization unit and between the zonequantization width and the interval of the quantization representativevalue. Because of the correlative relationship, an interval of thequantization representative values is also increased with an increase ofthe zone quantization width. In the present invention, the value to besubjected to quantization processing is obtained from the pixel valuesbeing included in the same zone. It is desired that the accuracy of thequantization should be higher (the interval of the quantizationrepresentative values should be narrower). However, there is a necessityof lowering the accuracy of the quantization so as to convert the valueto be quantized to a code having a predetermined bit length when thevalue to be quantized is large. In such case, the accuracy of thequantization is lowered, that is, the interval of the quantizationrepresentative values is enlarged. The zone quantization width can be anindex of quantization accuracy possible. The zone quantization width andthe related accuracy of the quantization (or the interval of thequantization representative values) may be determined in considerationof the intended bit length of a code, which is also related to the zonequantization width.

In this example, the interval of the quantization representative valuesis set to 4, that is, 2 bits of the lowest-order of the value to bequantized are rounded, so as to obtain the compressed encoded pixelvalue data of 8-bit length.

In the CODEC 13 of the present invention, the difference value itself isnot quantized, but the “value to be quantized” is calculated first basedon the target pixel value and its left side nearby same color pixelvalue, and the value to be quantized thus obtained is quantized and thequantized value is outputted as the compressed encoded pixel value data.

Values in this example, such as the target pixel value, the differencevalue, the compressed encoded pixel value data, and values used in theprocessing are shown in table 2.

TABLE 2 Pixel G1 R1 G2 R2 Pixel value g1 r1 g2 r2 Pixel value 2000 2502100 50 (in decimal notation) Pixel G2 R2 G3 R3 Pixel value    g2    r2   g3    r3 Pixel value 2100 50 1700 100 (in decimal notation)Difference Δg2 Δr2 Δg3 Δr3 value Difference 100 −200 −400 50 value (indecimal notation) Selected class &D(2) (&B(10)) Quantization 4(=2{circumflex over ( )} (10 − 8)) width Quantization 10 bit accuracy(12 − (10 − 8)) accuracy (8-bit quantization of 10-bit length data)Quantization 512 (=2{circumflex over ( )} (10 − 1)) reference value(=2{circumflex over ( )} (Quantization accuracy − 1)) Offset 1488 −2621588 −462 candidate value (Left side nearby same color pixel value −quantization reference value) Sign of offset + − + − candidate valueOffset value 1488 0 1588 0 Value to be 612 50 112 100 quantized (indecimal notation) (=target pixel value − offset value) Value to be0011100100 0000110010 0001110000 0001100100 quantized (in binarynotation) Quantized value 00111001 00001101 00011100 00011001 (in binarynotation) (Compressed encoded pixel value data)

First, the quantization processing will be explained, with a calculationof the compressed encoded pixel value data of the pixel value data ofthe pixel G2 taken as an example. The quantization width of the firstzone is 4. (By rounding 2 bits of the lowest-order of the data, the8-bit length data is obtained from the 10-bit length data. Therefore,the data regarding the pixel value of 12-bit length is rounded by 2-bitand is quantized with 10-bit accuracy.) In this case, the “quantizationreference value” (=(quantization accuracy−1)-th power of 2) is 512 (stepS108 a of FIG. 5B). That is, the quantization reference value isuniquely determined from the quantization accuracy, which is defined bythe zone quantization width being determined based on the maximum pixelvalue difference. Specifically, the quantization reference value isdetermined based on the maximum pixel value difference. Note that thequantization reference value may be a value which is greater than orequal to the maximum pixel value difference. Next, this quantizationreference value is subtracted from the pixel value g1, which is the leftside nearby same color pixel value of the pixel value g2, to obtain the“offset candidate value” (step S108 b of FIG. 5B). The sign of theoffset candidate value is checked. When the sign is non-negative, thisoffset candidate value is set as the “offset value” as it is. When theoffset candidate value is negative, the offset value is set to 0 (stepsS108 c, S108 d, and S108 e of FIG. 5B). Next, the offset value issubtracted from the target pixel value g2 to obtain the “value to bequantized” (step S108 f of FIG. 5B). This operation can also bedescribed that a sum of the difference value Δg2 and the quantizationreference value is set as the “value to be quantized”. As to the pixelG2, this value is &B(0011100100). 2 bits of the lowest-order of thisvalue to be quantized is rounded, and the quantized value (compressedencoded pixel value data) &B(00111001) is obtained (step S108 g of FIG.5B). When the highest-order bit in the rounded bits is “1”, the valuewhich is obtained by adding “1” to the lowest-order bit of the notrounded bits is set as the compressed encoded pixel value data. As amatter of course, when a carry to an upper digit occurs by addition ofthe “1”, other bits are also changed. Note that when the offsetcandidate value is negative, the offset candidate value may be obtainedby assigning all digits of the offset candidate value to zero except ther bits of the lowest-order of the offset candidate value. Here, r may beequal to the rounded bit number by quantization. For example, when thezone quantization width is 4, 2-bits are rounded in quantization, andtherefore the value obtained by assigning all bits of the offsetcandidate value to zero except the r bits of the lowest-order such as&B(0000000010) may be set as the offset value and the processingthereafter may be performed using the offset value as thus obtained. Inthis case, the value to be quantized of the pixel R2 is &D(48).

Next, explanation will be given, with the quantization of the pixelvalue r2 of the pixel R2 taken as an example. The left side nearby samecolor pixel value of r2 is r1. Hence, the offset candidate value is−262. When the offset candidate value is negative, the offset value is0. Therefore, the value to be quantized is the pixel value r2 itself of10-bit length, from which 2 bits of the upper-order of the pixel valuer2 are excluded, and the value thus obtained is &B(0000110010). 2 bitsof the lowest-order of this value to be quantized are rounded, and thequantized value (compressed encoded pixel value data) &B(00001101) isobtained. Similarly, the compressed encoded pixel value data&B(00011100) of the pixel value g3 of the pixel G3 and the compressedencoded pixel value data &B(00011001) of the pixel value r3 of the pixelR3 are obtained.

Referring to FIGS. 6 and 7, used quantities in the processing of thequantization unit 47 and processing steps are described. FIG. 6 is agraph plotting a relation between the pixel values g1, r1, g2, r2, g3,and r3 being used in this example and other quantities being also usedin this example. The pixel value is an integer value from 0 up to 4095,which is given as 12-bit data. In the quantization of the pixel valueg2, the offset value of g2 is obtained based on the pixel value g1 andthe quantization reference value, and the value obtained by subtractingthe offset value of g2 from the pixel value g2 is set as the value to bequantized of g2. The value to be quantized of g2 is also equal to thesum of the quantization reference value and the difference value Δg2.Therefore, the value to be quantized of g2 can also be obtained withoutusing the pixel value g1 and the pixel value g2. In the quantization ofthe pixel values r2 and r3, the offset value is 0. Therefore, the valuesto be quantized of r2 and r3 are equal to the pixel values r2 and r3. Inthe quantization of the pixel value g3, the offset value of g3 isobtained based on the pixel value g2 and the quantization referencevalue, and the value obtained by subtracting the offset value of g3 fromthe pixel value g3 is set as the value to be quantized of g3. The valueto be quantized of g3 is also equal to the sum of the quantizationreference value and the difference value Δg3. Therefore, the value to bequantized of g3 can also be obtained without using the pixel values g2and g3.

FIG. 7 is a graph showing details of the quantization processing, withthe pixel value g2 taken as an example. The values to be quantized ofall pixel values are the values in a range from 0 to twice thequantization reference value (the value of less than or equal to&D(1024) in this example). In the quantization, the value to bequantized of g2 is rounded with the quantization width 4. Therefore, thequantized value of g2 is &D(153). The value expressed in an 8-bit binarynotation such as &B(00111001) is the compressed encoded pixel value dataof the pixel G2. FIG. 7 shows an example of rounding up and roundingdown of the value to be quantized. For example, when the value to bequantized is in a range from &D(602) to &D(605), the quantized value is&D(151), and when the value to be quantized is in a range from &D(598)to &D(601), the quantized value is &D(150). Thus, when the highest-orderbit in the rounded bits is “1”, the number is rounded-up, and when thehighest-order bit in the rounded bits is “0”, the number isrounded-down. Note that in the quantization in the quantization unit 47,a simple bit shift (performing rounding down the every data) may beperformed.

The quantized value of each pixel value thus obtained is sent to thepacking unit 51 as the compressed encoded pixel value data of 8-bitlength, and is used for the generation of packing data (step S108 h ofFIG. 5B) and output of packing data (step S108 i of FIG. 5B). As topixel values being included in the second or later zone, the packingdata can be used in the quantization processing as the left side nearbysame color pixel value, by sending the decoded pixel value data which issent to the decoding unit 33 from the packing unit 51 and decodedtherein, to the pixel value storage unit 39 and stored therein, and bysending this data to the quantization unit 47.

<Generation of a Class Value Code in the Class Value Code GenerationUnit 49>

In the class value code generation unit 49, the class value is sent tothe packing unit 51 as the data of m-bit length, based on the zonequantization width having received. In this embodiment, m=2. When thezone quantization width is 2, the class value data of 2-bit lengthcorresponding thereto is sent to the packing unit 51. The bit length mof the class value data is not limited to 2. In this embodiment, fourkinds of class values are used, and hence, the m is set as m=2. Whenmore kinds of class values are required, m may be set to a largerinteger value. The class value is an index of the accuracy of thequantization (or the interval of the quantization representativevalues).

<Packing of the Data in the Packing Unit 51>

The initial pixel value data, the compressed encoded pixel value data,and the class value data being the quantization width information data,which are sent to the packing unit 51, are immediately sent to anunpacking unit 53, and decoded in the decoding unit 33. Those pieces ofthe decoded data can be used in the compression encoding of later targetpixel values. In addition, those data are packed into data to adjust itsdata width to s-bits, being an integral multiple of a unit for accessing(such as 32-bits) to access to the SDRAM 15, and the data can be sent tothe SDRAM 15 as the packing data of s-bit length.

Here, mainly, packing of the data in consideration of accessing to theSDRAM 15 will be explained.

One packing data has an s-bit length. Preferably, this bit length s isintegral multiple of 8.

In this embodiment, at least one pixel value for each color of pixels isrecorded in each packing data as the initial pixel value data. In thisexample, the pixel values g1 and r1, which are the pixel value data ofd(d=12) bit length inputted to the CODEC 13, are recorded as they are,as the initial pixel value data. The class value of m(m=2) bit lengthindicating the zone quantization width is also recorded in the packingdata, and further the compressed encoded pixel value data of the pixelwhich is included in this zone is recorded as the data of n(n=8) bitlength. The number of pixels which are included in one zone, p,satisfies p=4, and hence, pieces of the compressed encoded pixel valuedata of the pixels G2, R2, G3, and R3 are recorded as the data of 8-bitlength, respectively. Further, the class value of the next zone andpieces of the compressed encoded pixel value data of pixels beingincluded in the next zone can also be recorded.

The aforementioned several kinds of data are collected into one packingdata so that a total data length of the packing data does not exceeds-bits. When the total data length does not reach s-bits in the packingprocessing, predetermined dummy data is added to excess bits, and thepacking data of s-bits may be prepared. The packing data is sent to theSDRAM 15 and is stored therein.

In the present embodiment, for every packing data, the pixel value dataof the pixel of each color recorded first is recorded as the initialpixel value data. This improves convenience in reading the compressedencoded data. When data is read out from the data which is recorded inthe SDRAM 15 and constitutes image data of one frame image and if thedata being desired to be read out is only a part of the data of pixelvalues which constitutes the one frame image, the recording methodologymentioned above makes it possible to decode the required pixel valuesonly by reading out the data of s-bit length which includes the data ofthe required pixel and pixels of its vicinity. Therefore, there is nonecessity for reading data of all pixels which constitute one frameimage, thus making it possible to realize a high-speed processing andpower saving.

When the compressibility of the compressed encoded image data isprioritized over an effect described above, frequency of the pixel valuedata recorded as the initial pixel value data may be reduced. In thiscase, the initial pixel value data is not necessarily required to berecorded in one packing data. In addition, in order to improve thecompressibility, it is sometimes effective to set the number of thepixels included in one zone, p, larger. Thus, a frequency of appearanceof the class value data can be reduced. In addition, it is alsoeffective to set the bit length of the compressed encoded pixel valuedata, n, smaller.

In addition, the processing with a constant quantization width is alsopossible. In this case, the class value data indicating the quantizationwidth may not be recorded.

<Decoding in the Decoding Unit 33>

FIGS. 8A and 8B are flowcharts of the processing for the decoding. FIG.8B is a flowchart showing detailed steps in step S206 of FIG. 8A. Withreference to FIGS. 8A and 8B, explanation will be given hereunder to thedecoding processing for the compressed encoded data by the decoding unit33. The unpacking unit 53 inputs each kind of data, namely, the initialpixel value data, the compressed encoded pixel value data, and the classvalue being individually sent from the packing unit 51, and the packingdata, namely, the compressed encoded image data inputted from the SDRAM15, and, as to the packing data, separates the packing data intoindividual data (step S201 of FIG. 8A). The separated each kind of datais sent from the unpacking unit 53 to the inverse quantization unit 55at a predetermined timing.

The data being sent to the inverse quantization unit 55 is subjected tothe reversed processing with respect to the processing in thequantization unit 47. The decoded pixel value data is calculated fromthe compressed encoded pixel value data, and then, the decoded pixelvalue data is sent to the output unit 57. Specifically, the initialpixel value data is sent to the output unit 57, as it is, as the data ofd (=12) bits (steps S202 and S203 of FIG. 8A). In addition, this data isstored temporarily for use in later processing. The data of the classvalue, which is the quantization width information data, is storedtemporarily for use in the decoding processing of later compressedencoded pixel value data (steps S204 and S205 of FIG. 8A). In step S206,the decoding processing is performed and the decoded pixel value data isgenerated. The quantization width and the quantization reference valueare obtained from the class value data (steps S206 a and S206 b of FIG.8B). Then, the offset candidate value is obtained, which is a differencebetween the initial pixel value data being previously processed and thequantization reference value or a difference between the decoded pixelvalue data and the quantization reference value (step S206 c of FIG.8B). When the offset candidate value is zero or less, the offset valueis set to zero (steps S206 d, S206 e, and S206 f of FIG. 8B). Thecompressed encoded pixel value data is bit shifted (inversely quantized)based on the zone quantization width shown by the class value togenerate the inversely quantized pixel value data (step S206 g of FIG.8B). And by obtaining the sum of the inversely quantized pixel valuedata and the offset value which is a difference between the previouslyprocessed initial pixel value data and the quantization reference valueobtained from the class value or a difference between the decoded pixelvalue data and the quantization reference value and the inverselyquantized pixel value data, the pixel value data is decoded to a data ofd=12 bit length (step S206 h of FIG. 8B). Then the decoded data is thensent to the output unit 57 as the decoded pixel value data (step 5206 ofFIG. 8A). In the quantization unit 47, rounding up or down is performedin the quantization, however, in the inverse quantization unit 55, onlya simple bit shift may be performed.

The decoded pixel value data thus obtained is sent to the encoding unit31 and the YC processing unit 17 at respective predetermined timings.

When the quantization processing is performed with a constantquantization width, the class value data is not necessary. In this case,the decoding processing may be performed with a given quantizationwidth.

Note that, in this embodiment, the image pickup element having the Bayerarray is used. However, the CODEC according to the present invention canbe used in combination with the image pickup element having other array,for example, the image pickup element having a primary color verticalstripe array, RGBRGBRGB . . . . Also, the CODEC according to the presentinvention can be used even in a case that a sensor (pixel) array of theimage pickup element is a complementary color checked array, such ashaving CyYeCyYeCyYe . . . . The CODEC according to the present inventioncan be used, irrespective of an array format of the pixels in the lightreceiving unit of the image pickup element. Not only a case that thepixel array is a tetragonal lattice array, but also a case that thearray format of the pixels is a honeycomb-like array, the

CODEC according to the present invention can be used.

Embodiment 2

<CODEC with Feedback Processing of Rrror Due to Quantization>

This embodiment provides a digital still camera (DSC) provided with aCODEC 113 having error feedback processing for reducing an error. Thiserror often occurs in the quantization of the pixel value data by theCODEC (a difference between the pixel value data and the decoded pixelvalue data obtained through applying compression encoding processing tothe pixel value data and further applying the decoding processing to theencoded pixel value data).

The DSC of this embodiment is similar to the DSC 1 according to thefirst embodiment except for the processing by the CODEC 113. Accordingto FIG. 1, it is appropriate to consider that the CODEC 113 as will beexplained hereunder is mounted on the DSC of this embodiment instead ofthe CODEC 13. Here, the configuration and processing of the CODEC 113will be explained. A part not described in particular may be the same asthat of the first embodiment.

FIG. 9 is a block diagram of the CODEC 113 according to this embodiment.The CODEC 113 is different from the CODEC 13 according to the firstembodiment in the point that an error detection unit 59, a coefficientmultiplication unit 61, an integer conversion unit 63, and an errorcorrection unit 65 are added to the encoding unit 31, and the point thatthe output of the output unit 57 of the decoding unit 33 is inputted tothe error detection unit 59 and the quantization unit 47.

The error feedback processing by the CODEC 113 will be explainedhereunder. Here, the error feedback processing in the quantization ofthe pixel value r3 of the pixel R3 is taken as a specific example.

The pixel value r3 is inputted from the ADC 11 to the target pixel valueinput unit 35, which is similar to the processing in the firstembodiment. The pixel value r3 is a pixel value data to be subjected tothe compression encoding processing and then be recorded as thecompressed encoded pixel value data. Therefore, the pixel value r3 issent to a difference generation unit 37. At the same time, the pixelvalue r2, being the left side nearby same color pixel value of thetarget pixel value r3, is sent from the storage unit 39 to thedifference generation unit 37. The difference value Δr3=r3−r2 isobtained. A difference quantization range of the pixel R3 is obtained.The zone quantization width is determined based on the differencequantization range of the pixels being included in the same zone as thatof the pixel R3 by the zone quantization width determination unit 43.Then, the zone quantization width thus determined is sent to thequantization unit 47.

For the quantization of the pixel value r3 in the quantization unit 47,the target pixel value r3 and the left side nearby same color pixelvalue r2 are necessary as well as the aforementioned zone quantizationwidth. In this embodiment, an error corrected target pixel value Cr3processed in the error correction unit 65 is inputted to thequantization unit 47 in place of the target pixel value r3. In addition,a decoded pixel value is used as a left side nearby same color pixelvalue in the quantization of the pixel value r3.

According to the table 2, the pixel value r2 is &D(50). Meanwhile, thedecoded pixel value is &D(52). In the CODEC 113 of this embodiment whichuses the decoded pixel value as the left side nearby same color pixelvalue for quantization, the error which has occurred in the quantizationof the pixel value of a preceding pixel has an influence on thequantization of the pixel value of later same color pixel. The errorfeedback processing of the CODEC 113 as will be described hereunder isexecuted for the purpose of reducing such influence.

The decoded pixel value being outputted from the output unit 57 of thedecoding unit 33 is sent to the error detection unit 59 of the encodingunit 31. The pixel value r2 being recorded in the storage unit 39 isalso inputted to the error detection unit 59 with a matched timing tocalculate an error. Here, the error Er2 which is an error between thepixel value r2 (true value) and the decoded pixel value are obtainedfrom the difference between the true pixel value and the decoded pixelvalue. That is, Er2=(true pixel value)−(decoded pixel value)=(−2).

The error Er2 with respect to the pixel value r2 is sent to thecoefficient multiplication unit 61. A value αEr2 is obtained bymultiplying the error by a coefficient α, namely, α·Er2 (=αEr2). Thecoefficient α may be a value obtained from an experiment.

In this example, α is set to be 0.50 for simplification. In thisexample, hence, the αEr2 is (−1.00). Generally, α is a real numberobtained from an experiment, and therefore, a decimal fraction partappears in the error αEr2, which is the product of Er2 and α. Hence, theinteger conversion unit 63 converts the real number αEr2 into aninteger, so that the integer value thus obtained is set as the errorcorrection value. An integer conversion processing by the integerprocessing unit 63 may be a general processing. For example, theprocessing such as rounding-down of the fraction part, rounding-off ofthe first order of the fraction part, or rounding-up of the fraction maybe performed. In this example, the error correction value is (−1).

Next, the processing in the error correction unit 65 will be explained.The target pixel value r3 and the error correction value for the pixelvalue r3, (−1) are inputted to the error correction unit 65. Here, theerror correction value is given to the target pixel value as a negativefeedback. That is, the error corrected target pixel value Cr3 is equalto the value obtained by subtracting the error correction value from thetarget pixel value. That is, Cr3=r3−(−1)=r3+1. The error correctedtarget pixel value Cr3 is inputted to the quantization unit 47 and isthen quantized based on the decoded pixel value Dr2 and the zonequantization width. The quantized value is outputted to the packing unit51 as the compressed encoded pixel value data.

The processing in the quantization unit 47 may be similar to that of thefirst embodiment. A value for the zone quantization width is determinedon the basis of the difference value between the target pixel value andthe left side nearby same color pixel value which is stored in thestorage unit 39 and does not include an error. The error correctedtarget pixel value which is corrected by the error correction unit 65 isused for the target pixel value. As the left side nearby same colorpixel value in the quantization unit 47, the decoded pixel value datawhich has been subjected to the compression encoding processing andfurther subjected to the decoding processing is used.

Embodiment 3

<Use of a Variable Bit Length Compressed Encoded Pixel Value Data Basedon a Distribution Ratio>

This embodiment provides a digital still camera (DSC) of which CODEC iscapable of changing and optimizing the bit length of the compressedencoded pixel value data (the number of bits of the compressed encodedpixel value data), which is the compression encoding value of the pixelvalue data, according to the characteristic of an object which isactually imaged. The present embodiment provides the CODEC 213 havingfurther higher compression efficiency. The DSC of this embodiment issimilar to the DSC of the first and second embodiments, except for theprocessing in the CODEC 213. According to FIG. 1, it is appropriate toconsider that the CODEC 213 as will be explained hereunder is mounted onthe DSC of this embodiment instead of the CODEC 13. Here, theconfiguration and processing of the CODEC 213 will be explained. A partnot described in particular may be the same as that of the firstembodiment. The CODEC 213 of the present embodiment performs an optimalbit length of the compressed encoded pixel value data etc. determinationprocessing. The optimal bit length of the compressed encoded pixel valuedata etc. determination processing determines the optimal number of bitsof compressed encoded pixel value data, quantization width, and the likeon the basis of optical information with respect to the object, whichhas entered the image pickup element before the moment when an imagepickup instruction is given (the moment when a shutter button is pressedby an operator).

<Configuration of CODEC 213>

FIG. 10 is a block diagram of the CODEC 213 according to the presentembodiment. The CODEC 213 is different from the CODECs 13 and 113according to the first and second embodiments in the point that adifference quantization range extraction unit 67 and a distributionratio analysis unit 69 are added into the encoding unit 31. Regardingthe other configurations, the same configuration as the CODEC 13 or theCODEC 113 may be provided to the CODEC 213. That is, FIG. 9 shows thatthe CODEC 213 has a similar configuration as that of the CODEC 13 exceptfor the aforementioned point. However, the CODEC 213 may have thesimilar configuration as that of the CODEC 113 except for theaforementioned point.

The difference quantization range extraction unit 67 inputs the outputfrom the difference quantization range determination unit 41, namely,the quantization range of each pixel difference value, and stores theinput temporarily therein. A plurality of quantization ranges stored inthe extraction unit 67 is outputted to the distribution ratio analysisunit 69 at a predetermined timing.

The distribution ratio analysis unit 69 analyzes the quantization rangesof thinned pixels which correspond to one frame image, and determinesthe optimal number of bits of the compressed encoded pixel value data.The optimal number of bits (t [bit]) of the compressed encoded pixelvalue data being thus determined is sent to the quantization unit 47 andthe class value code generation unit 49.

In the quantization unit 47, the compressed encoded pixel value data oft-bit length is generated. The class value code generation unit 49generates a class value code of m-bits expressing the zone quantizationwidth being inputted from the zone quantization width determination unit43 and the optimal number of bits of the compressed encoded pixel valuedata, which is then outputted to the packing unit 51. The class value inthe present embodiment may be generated according to a predefinedconversion rule so as to include the optimal number of bits of thecompressed encoded pixel value data.

<Processing of Determining the Optimal Number of Bits of the CompressedEncoded Pixel Value Data in the CODEC 213>

The DSC 1 has an operation mode called a “monitor” mode. The monitormode is a mode of operation in which the optical information (such as anobject image) being incident to the image pickup element 5 is displayedon the display unit 27, for example, so that an operator can decide anobject to wait for a timing to click a shutter. In the monitor mode, theDSC1 operates processing only to the pixels of predetermined ratio,instead of extracting accumulated charges of all pixels of the imagepickup element 5 and applying signal processing to all extractedaccumulated charges. Such image pickup element drive mode is called adraft mode or a thinning mode. That is, in the monitor mode, the DSC 1drives the image pickup element 5 in the draft mode. The DSC 1 respondsto a shutter button operation by the operator and switches the imagepickup element drive mode from the draft mode to an all pixel drivemode, so that imaging is performed.

In the monitor mode, the CODEC 213 inputs pixel values in the draftmode, that is, the CODEC 213 inputs the thinned number of pixel values.Thinning of the pixels is performed in a predetermined pattern, andtherefore, the left side nearby same color pixel value with respect tothe target pixel value in the draft mode can be defined in accordancewith a predetermined rule, although the rule has to be modified from therule at the imaging. In the difference generation unit 37, thedifference value is calculated for each target pixel value by using theleft side nearby same color pixel value with respect to the target pixelvalue thus defined, and the difference quantization range determinationunit 41 determines the difference quantization range with respect toeach target pixel value. The difference quantization range thusdetermined is sent to the difference quantization range extraction unit67.

The difference quantization range extraction unit sends the differencequantization ranges for a recent one frame image to the distributionratio analysis unit 69, utilizing the difference quantization ranges foreach target pixel which is temporarily stored.

<Processing in the Distribution Ratio Analysis Unit 69>

The distribution ratio analysis unit 69 analyzes the differencequantization ranges having received and determines the optimal number ofbits of the compressed encoded pixel value data.

The optimal number of bits of the compressed encoded pixel value data isthe bit length of the compressed encoded pixel value data which isgenerated by the CODEC 213 for recording the pickup image in the SDRAM15 as compressed encoded data. In the first and second embodiments, thisbit length n is fixed at 8-bit. However, in some cases, a sufficientimage quality can be obtained with the bit length being set shorter than8-bit length, depending on the object to be imaged. Therefore, in thisembodiment, an optimal bit length is determined based on objectinformation which has been retrieved in the monitor mode.

First, the distribution ratio analysis unit 69 analyzes the differencequantization ranges of the recent one frame image including the object,and obtains an appearance distribution of the difference quantizationranges. Then, for example, the distribution ratio analysis unit 69obtains the distribution ratios of the difference quantization rangeeach included in the value which is less than or equal to the naturalnumber (n−1(=8−1=7), namely, a natural number v, v−1, v−2, or, . . . .From those distribution ratios, a maximum difference quantization rangew of which distribution ratio is greater than or equal to apredetermined threshold value is obtained. The number obtained by adding1 to the maximum difference quantization range w is set as the optimalnumber of bits of the compressed encoded pixel value data, which is thensent to the quantization unit 47 and the class value code generationunit 49.

For example, when the appearance ratio of the difference quantizationrange having the value of 4 or less is a predetermined ratio (90%, forexample) with respect to all quantization range, it can be estimatedthat about the 90% of the pieces of the pixel value data can be recordedwithout rounding by the quantization unit 47 using the data of 4+1=5-bitlength. It is disadvantageous in an aspect of the compressibility, forsuch an object, to assign 8-bit length to the number of bits of eachcompressed encoded pixel value data. In such case, the CODEC 213 iscapable of improving the compressibility without quality deterioration.In such case, the quantization unit 47 outputs compressed encoded pixelvalue data which has 5-bit length for each target pixel value. As toclass value, the class value having the information regarding theoptimal number of bits of the compressed encoded pixel value data isgenerated. Therefore, in decoding in the decoding unit 33, the number ofbits of each compressed encoded pixel value data is determined based onthe class value, and decoding processing similar to the foregoingembodiment can be performed.

Note that in the distribution ratio analysis unit 69, instead ofanalyzing the difference quantization range of the recent one frameimage, only a part of the image, for example an important part or acentral region in the image is analyzed and the optimal number of bitsof the compressed encoded pixel value data may be determined.

In addition, in the former example, although the predetermined ratio isset to 90%, a ratio of any of 1 to 100% may be used as a thresholdvalue. Further, by examining the characteristic of the object, and basedon the result of such examination, the threshold value may be determinedflexibly for each imaging processing.

In addition, the difference quantization ranges being included in oneframe image may be divided into a plurality of blocks and, in eachblock, the optimal number of bits of the compressed encoded pixel valuedata may be determined. In this case, the data of different bit lengthsis mixed in the compressed encoded pixel value data of one frame image.However, the class values are stores such variation of the numbers ofbits of these compressed encoded pixel value data. Therefore, correctdecoding is possible in the decoding unit 33.

In addition, even in the monitor mode, the image pickup element 5 may bedriven in an all pixels driving mode.

In addition, in all embodiments of the present invention, a function forstoring or temporarily storing data may be realized by using not only astoring element but also the known delay circuit.

INDUSTRIAL APPLICABILITY

The present invention can be employed in a general electronic apparatusor a general video apparatus which have a CCD sensor or an MOS sensor.Those apparatuses are considered to be further broadly used hereafter.The present invention can compress a data volume with giving almost nodamage to an image quality, despite having a simple structure.

By reducing the number of bits of recorded data, a use amount of amemory and an access amount of a memory can be reduced, and a cost andpower consumption can also be reduced.

The present invention is expected to be used in further purposes in thedigital still camera and a camera for photographing a moving image.

1. A digital data encoding device which receives pixel value data in adigital format indicating a signal from a light receiving unit in whichat least a pixel for sensing a first color and a pixel for sensing asecond color are periodically arranged, and processes the pixel valuedata, comprising: a difference generation unit that outputs a differencevalue between first pixel value data from a first pixel which senses thefirst color and second pixel value data from a second pixel which sensesthe first color being positioned in the vicinity of the first pixel as afirst pixel difference value, and outputs a difference value of thirdpixel value data from a third pixel which senses a second color andfourth pixel value data from a fourth pixel which senses the secondcolor being positioned in the vicinity of the third pixel, as a secondpixel difference value; a quantization reference value determinationunit that obtains a maximum value between an absolute value of the firstpixel difference value and an absolute value of the second pixeldifference value as a maximum pixel value difference, and determines avalue being greater than or equal to the obtained maximum pixel valuedifference as a quantization reference value; an offset value settingunit that sets a difference between the first pixel value data and thequantization reference value as a first offset value and sets adifference between the third pixel value data and the quantizationreference value as a second offset value; a value to be quantizedsetting unit that sets the difference between the second pixel valuedata and the first offset value as a first value to be quantized andsets the difference between the fourth pixel value data and the secondoffset value as a second value to be quantized; and a quantization unitthat quantizes the first value to be quantized and the second value tobe quantized and obtains first compressed encoded pixel value data andsecond compressed encoded pixel value data.
 2. The digital data encodingdevice according to claim 1, wherein the first color and the secondcolor are different from each other.
 3. The digital data encoding deviceaccording to claim 1, further comprising an offset value zero resettingunit that resets the offset value to zero when the offset value beingdefined by said offset value setting unit is less than or equal to zero.4. The digital data encoding device according to claim 1, furthercomprising a quantization width determination unit that determines aquantization width of the quantization.
 5. The digital data encodingdevice according to claim 4, wherein the quantization width is increasedas the maximum pixel value difference becomes larger.
 6. The digitaldata encoding device according to claim 4, comprising a quantizationwidth information data generation unit that encodes the determinedquantization width into a code of m-bit length, the m being a naturalnumber.
 7. The digital data encoding device according to claim 4,wherein said quantization width determination unit determines thequantization width to any one of preliminarily determined pluralquantization widths.
 8. The digital data encoding device according toclaim 7, wherein the number of the plural quantization widths is lessthan or equal to m-th power of
 2. 9. The digital data encoding deviceaccording to claim 6, wherein the m is
 2. 10. The digital data encodingdevice according to claim 6, further comprising a compressed encodedimage data generation unit, wherein said compressed encoded image datageneration unit generates compressed encoded image data of s-bit length,the s being a natural number, including at least one of the quantizationwidth information data, the first compressed encoded pixel value data,and the second compressed encoded pixel value data and, the s being amultiple of
 8. 11. The digital data encoding device according to claim10, wherein said compressed encoded image data generation unit recordsthe first pixel value data as they are in the compressed encoded imagedata as initial pixel value data.
 12. The digital data encoding deviceaccording to claim 7, wherein said quantization width determination unitobtains a maximum value between the numbers of digits required for anunsigned integer binary value notation of each of the first pixeldifference value and the second pixel difference value so as todetermine the quantization width from the plural quantization widths.13. The digital data encoding device according to claim 1, furthercomprising: an error correction unit that corrects the second pixelvalue data and generates error corrected pixel value data; and a digitaldata decoding unit that decodes compressed encoded pixel value data andoutputs decoded pixel value data, wherein: said offset value settingunit sets the first offset value by using the decoded pixel value datainstead of the first pixel value data; and said value to be quantizedsetting unit sets the first value to be quantized by using the errorcorrected pixel value data instead of the second pixel value data. 14.The digital data encoding device according to claim 13, wherein thecorrection of the second pixel value data by said error correction unitis performed by subtracting a correction value related to a differencebetween the first pixel value data and the decoded pixel value data fromthe second pixel value.
 15. A digital data decoding device, comprising:a compressed encoded image data input unit that inputs compressedencoded image data of s-bit lengths, the s being a natural number, whichhas an initial pixel value data part in which first pixel value data ofa first pixel which senses a first color are recorded as they are asfirst initial pixel value data and a compressed encoded pixel value datapart in which first compressed encoded pixel value data indicatingsecond pixel value data of a second pixel which senses the first colorbeing positioned in the vicinity of the first pixel are recorded; anoffset value setting unit that obtains a difference between the firstinitial pixel value data and a first quantization reference value beingset as a first offset value; an inverse quantization unit that inverselyquantizes the first compressed encoded pixel value data by using a firstquantization width being set to obtain first inversely quantized pixelvalue data; and a decoded pixel value generation unit that obtains a sumof the first inversely quantized pixel value data and the first offsetvalue to generate first decoded pixel value data.
 16. The digital datadecoding device according to claim 15, wherein: third pixel value dataof a third pixel which senses a second color being positioned nearby thefirst pixel is further recorded as they are as second initial pixelvalue data in the initial pixel value data part; second compressedencoded pixel value data which indicates fourth pixel value data of afourth pixel which senses the second color being positioned in thevicinity of the third pixel is further recorded in the compressedencoded pixel value data part; said offset value setting unit obtains adifference between the second initial pixel value data and the firstquantization reference value as a second offset value; said inversequantization unit further inversely quantize the second compressedencoded pixel value data by using the first quantization width so as toobtain second inversely quantized pixel value data; and said decodedpixel value generation unit further obtain a sum of the second inverselyquantized pixel value data and the second offset value to generatesecond decoded pixel value data.
 17. The digital data decoding deviceaccording to claim 15, wherein the first color and the second color aredifferent from each other.
 18. The digital data decoding deviceaccording to claim 16, wherein: third compressed encoded pixel valuedata indicating fifth pixel value data of a fifth pixel which senses thefirst color being positioned in the vicinity of the second pixel andbeing nearby the fourth pixel is further recorded in the compressedencoded pixel value data part; said offset value setting unit furtherobtains a difference between the first decoded pixel value data andsecond quantization reference value being set as a third offset value;said inverse quantization unit further inversely quantizes thirdcompressed encoded pixel value data by using the second quantizationwidth being set to obtain third inversely quantized pixel value data;and said decoded pixel value generation unit further obtains a sum ofthe third inversely quantized pixel value data and the third offsetvalue to generate third decoded pixel value data.
 19. The digital datadecoding device according to claim 15, further comprising an offsetvalue zero resetting unit that resets an offset value to zero when theoffset value being defined by the offset value setting unit is less thanor equal to zero.
 20. The digital data decoding device according toclaim 18, wherein the compressed encoded image data includes aquantization width information data part in which at least one of firstquantization width information data having information regarding thefirst quantization width and second quantization width information datahaving information regarding the second quantization width is recorded.21. The digital data decoding device according to claim 20, furthercomprising a quantization width setting unit that sets a quantizationwidth of the inverse quantization to any one of the preliminarilydetermined plural quantization widths, wherein said quantization widthsetting unit sets the first quantization width and the secondquantization width to each one of the plural quantization widths,respectively, on the basis of the first quantization width informationdata and the second quantization width information data.
 22. The digitaldata decoding device according to claim 21, further comprising aquantization reference value setting unit that sets a quantizationreference value of the inverse quantization to any one of thepreliminarily determined plural quantization reference values, whereinsaid quantization reference value setting unit sets the firstquantization reference value and the second quantization referencevalue, respectively, on the basis of the first quantization widthinformation data and the second quantization width information data. 23.The digital data decoding device according to claim 22, wherein thefirst quantization width information data and the second quantizationwidth information data are pieces of data of m-bit lengths, the m beinga natural number, respectively.
 24. The digital data decoding deviceaccording to claim 23, wherein the numbers of the plural quantizationwidths and the plural quantization reference values are less than orequal to the m-th power of 2, respectively.
 25. The digital datadecoding device according to claim 23, wherein the m is
 2. 26. A digitaldata encoding method, for receiving pixel value data in a digital formatindicating a signal from a light receiving unit in which a pixel forsensing a first color and a pixel for sensing a second color areperiodically arranged and processing the pixel value data, comprising:generating and outputting a difference value between first pixel valuedata of a first pixel which senses a first color and second pixel valuedata of a second pixel which senses the first color and is positioned inthe vicinity of the first pixel as a first pixel difference value andgenerating and outputting a difference value between third pixel valuedata of a third pixel which senses a second color and fourth pixel valuedata of a fourth pixel which senses a second color and is positioned inthe vicinity of the third pixel as a second pixel difference value;obtaining a maximum value between an absolute value of the first pixeldifference value and an absolute value of the second pixel differencevalue as a maximum pixel value difference, determining a value beinggreater than or equal to the maximum pixel value difference, anddetermining the value as a quantization reference value; setting adifference between the first pixel value data and the quantizationreference value as a first offset value and setting a difference betweenthe third pixel value data and the quantization reference value as asecond offset value; setting a difference between the second pixel valuedata and the first offset value as a first value to be quantized andsetting a difference between the fourth pixel value data and the secondoffset value as a second value to be quantized; and quantizing the firstvalue to be quantized and the second value to be quantized to obtainfirst compressed encoded pixel value data and second compressed encodedpixel value data, respectively.
 27. A digital data decoding method,comprising: inputting compressed encoded image data of s-bit lengths,the s being a natural number, which has an initial pixel value data partin which first pixel value data of a first pixel which senses a firstcolor are recorded as they are as first initial pixel value data and acompressed encoded pixel value data part in which first compressedencoded pixel value data indicating second pixel value data of a secondpixel which senses the first color being positioned in the vicinity ofthe first pixel are recorded; obtaining a difference between the firstinitial pixel value data and first quantization reference value beingset as a first offset value; inversely quantizing the first compressedencoded pixel value data by using first quantization width being set toobtain first inversely quantized pixel value data; and obtaining a sumof the first inversely quantized pixel value data and the first offsetvalue to generate first decoded pixel value data.